DOCUMENT RESUME 



ED 332 250 



CS 507 459 



AUTHOR 
TITLE 

PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Krendl, Kathy A.; Broihier, Mary 

student Responses to Computers: A Longitudinal 

Study. 

May 91 

20p.; Paper presented at the Annual Meeting of the 
International Communication Association (41st, 
Chicago, IL, May 23-27, 1991). 
Reports - Research/Technical (143) — 
Speeches/Conference Papers (150) 

MP01/PC01 Plus Postage. 

"Computer Assisted Instruction; Computer Literacy; 
"Computers; "Educational Media; Elementary Secondary 
Education; Instructional Effectiveness; Longitudinal 
Studies; Media Research; Novelty (Stimulus 
Dimension); Public Schools; Student Attitudes 
"Computer Anxiety; "Educational Media Role; 
Tennessee 



ABSTRACT 

Using R« E. Clark's concept of media attributions, a 
study examined the evolution of fourth through tenth grade students' 
perceptions about computers on three dependent variables — preference 
perceived learning, and perceived difficulty, over the course of 3 
years. Subjects were 339 public school students in Tennessee who 
completed a self -administered questionnaire. Findings demonstrated 
clear evidence of novelty effects, students' judgments regarding 
preferences for computers declined significantly as did their 
perceptions of learning from the technology during the 3 years. 
Perceived difficulty of using computers, which was expected to 
decline, remained stable. In addition, both gender and age proved to 
be significantly related to all three dependent variables. Older 
students were consistently more skeptical about the technology than 
were younger students, and boys were consistently mere positive than 
girls. These relationships showed no evidence of change over the 
course of 3 years. The results support critiques of the 
methodological limitations of the dominant approach to the study of 
computer effects in learning environment. Reports of short-term 
experimental applications of the technology have led to misleading 
generalizations about the computer's instructional potential. 
(Thirty-three references are included.) (PRA) 
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Student Responses to Computers: 
A Longitudinal Study 

The results of this study support critiques of the methodological limitations of the dominant 
approach to the study of computer effects in learning environments. That is, reports of short-term 
experimental applications of the technology hive led to misleading generalizations about the 
computer's instructional potential. Using Clark's concept of media attributions, this study examines 
the evolution of students' (fourth through tenth grade) perceptions about computers on three 
dependent variables - preference, perceived learning, and perceived difficulty, over the course of 
three years. The findings demonstrate clear evidence of novelty effects. Students' judgments 
regarding preferences for computers declined significantly as did their perceptions of learning from 
the technology during the three years. Perceived difficulty of using computers, which was expected 
to decline, remained stable. In addition, both gender and age proved to be significantly related to all 
three dependent variables. Older students were consistently more skeptical about the technology than 
were younger students, and boys were consistently more positive than girls. These relationships 
showed no evidence of change over the course of three years. 
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Student Responses to Computers: 
A Longitudinal Study 

Critics ind reviewers of research on computers and learning nave identified important 
limitations characteristic of many studies. These criticisms have focused, to a large extent, on the 
inadequacies of the traditional experimental approach, in particular, the use of the media 
comparison model, to enhance knowledge about the instructional effectiveness of computers (Clark, 
1984; 1985; 1990; Clark & Salomon, 1985; Kulik, Kulik, & Cohen, 1980; Papert, 1987; Salomon 
& Gardner, 1986), as well as the glaring absence of longitudinal research efforts emphasizing the 
long-term impact of computers, as opposed to short-term novelty effects (Krendl & Lieberman, 
1986; Lieberman, 1985; Salomon, 1984). 

The central concern in these critiques is grounded in what Salomon (1984; Salomon & 
Gardner, 1986) has characterized as the naivety or simplemindedness of research which purports to 
draw valid and generalizable conclusions about the instructional effectiveness of "the computer" 
based on short-term, experimental applications of the technology. Typically these studies are vague 
about the details of the computer application being used - characteristics and content of the 
software, degree of student control, role of the teacher, description of the learning environment, 
and so on. In these studies researchers often discuss computer technology as if it were one 
monolithic system that could readily be transferred from one setting and one learning environment 
to another and could reasonably be expected to obtain the same results. 

Of course, challenges of this underlying assumption are frequently cited in critiques of the 

rapidly accumulating body of studies. Salomon has explained the situation as follows: 

Too many factors interact to produce any uniform effects. There are 
various modes of computer use, ranging from computer-based instruction 
to LOGO programming, and from data crunching to mindlessly playing 
games; there are various kinds of users, various contexts in which the 
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use takes place, and other variables of Importance that Interact. As there 
Is no capital letter "Computer," no two combinsUoas of these are likely 
to yield Identical effects (1914, p. t). 

In addition, Dark (1984; 1985; Clark & Salomon, 1985) has suggested that the retulte 
gleaned from research on computers and learning are highly suspect because they may well be 
attributable to a variety of uncontrolled effects, including novelty effects and differences in 
instructional method or instructional content, rather than to the delivery medium, the computer. In 
particular, discussions of the novelty effect have noted that high levels of motivation and attention 
are often associated with new activities. The critics argue that it Is the motivation and attention, 
inflated by the novelty of the computer application, that account for enhanced learning outcomes 
and positive attitudes that result, not the delivery system itself as researchers often conclude. The 
implication of this criticism is that the positive outcomes - learning from the new medium, having 
more positive attitudes about learning - will tend to decline as the technology becomes more 
familiar and its novelty wears off. However, the constraints of the short-term experimental 
applications prohibit any effort to test for the existence or power of novelty effras. The only 
evidence that speaks to the issue of the role of novelty effects appears in meta <tal>es comparing 
the relative strength of learning effects in longer versus shorter term computer applications (Knlik, 
Bangert, & Williams, 1983; Kulik & Kulik, 1987). Tnese studies conclude that short-term 
applications typically report more positive learning effects than do studies of long durstion. 

The general conclusion of the critiques is that research on computers and learning is weak in 
many critical areas - it lacks scientific rigor, conceptual development, appropriate and adequate 
research methods. Rather than learning from and building on research addressing the effects of 
previous new technologies, such as television, research on computers and learning imitates and 
replicates the limitations and naivety of such work (Clark, 1984; Salomon & Gardner, 1986). 



One of the suggestions emerging from the various critique of research on computers is to 
shift attention away from the examination of effects of the medium exclusively and to begin to 
address Individuals' approaches to, attributions about, and perceptions of the technology (Clark, 
1984; 1985; Clark & Salomon, 1985; Salomon & Gardner, 1986). Tbe purpose of the present 
study is to follow this suggestion using longitudinal data from a large, representative sample to 
study the evolution of responses to computers over an extended period of time. This approach 
should provide the researcher with evidence to assess the magnitude and substance of any sort of 
novelty effects related to computers, as well as understand bow the role of computers - users* 
perceptions of and attributions about them - have changed over time as the technology has become 
more firmly established in our culture and, in particular, in our schools. 

Hie particular dimensions of interest in the study of individuals' responses to computers are 
based on Clark's discussion of media attributes (Clark, 1983; Clark & Salomon, 1985). According 
to Clark, examination of media attributes is necessary to understand learning outcomes resulting 
from media applications. He has identified three dimensions - preference, difficulty, and 
perceived learning - as being fundamental to understanding individuals* media attributions. 

Formulating hypotheses for understanding individuals' perceptions of and attributions about 
computers over time required a review of two areas of inquiry - studies of media novelty effects 
and research on computer attitudes. Both literatures were relevant in shaping our expectations 
about how computer attributions might evolve over time. TTicse hypotheses focus on respondents' 
assessments of computers in relation to the three dimensions of interest - preference, difficulty, 
and perceived learning. In each case we identified a general trend we expected to find among the 
sample as a whole. In addition, we hypothesized changes we anticipated based on previous 
research examining the role of gender and grade level in determining students' responses to the 
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technology. Each set of hypotheses below Includes predictions for the general trend and then 
specifically for the Influence offender and grade level. 
Hypotheses rrfued to computer prefermi 

Two approach rs related to the assessment of preference for or enjoyment of a medium 
appear in the literature. One is an Indirect approach that equates this concept with direct time 
expenditures (Himmelweit, Oppenheim, k Vince, 1958; Schramm, Lyle, & Parker, 1961; Vitalari, 
Venkatesh, & Gronhaug, 1985). That is, researchers test for novelty effects by measuring how 
much time is spent on an activity and how time expenditures on other related activities change as a 
result. n>e other approach is to ask respondents directly to assess their level of enjoyment for the 
activity (Henke & Donohue, 1986). Regardless of the conceptualization used for preference, the 
conclusion from these studies is that, in general, level of enjoyment or preference for a media 
activity tends to decline as Its novelty wears off over time. 

ITius, in the case of computers, we expected individuals' attributions about how much they 
enjoy using computers to decline significantly over time. We also expected gender to play a 
significant role with girls' assessments of computer enjoyment being significantly lower than boys' 
over time (Abler * Sedlacek, 1987; Collia & Ollila, 1986; Griffin, Gillis, & Brown, 1986; 
Johnston, 1987; Koohang, 1989; Moore, 1985. In addition, based on previous work on the 
relationship between grade level and computer attitudes, we predicted that younger students' 
enjoyment of computers would be significantly higher than older students' < jyment over time 
(Smith, 1987). That is, we anticipated a negative relationship between grade level and computer 
enjoyment. 

Hypotheses related tn perceived diffic ulty of using cflfflBlUfia 

We expected perceived difficulty of computers to decline; over time as students gained 
experience and confidence with the technology. In general, researchers have found that students' 
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lava! of confidence with the technology bai locreited at a reault of practice and experience (Arndt, 
Clevenger, & Meiskey, 1985; DaJton & HannaAn, 1984; 1985; Griffin, Gillis, & Brown, 1986; 
Koohani, 1989; Moore, 1985; O'Toole & Wagner, 1985). We anticipated that gender would also 
pliy a significant role with girls' reports of difficulty being higher then boys' (Abler & Sedlacek, 
1987; Collis & Ollila. 1986; Koohang, 1989). Grade level was also expected to influence 
perceptions of difficulty. We anticipated that younger students would perceive the technology as 
less difficult than older students because of their generally we positive attitudes (Collis & Ollila, 
1986; Smith, 19K7). Thus, we hypothesize a positive relationship between grade level and 
perceived difficulty. 

Hypotheses related tn pei-r^vcd learning f rom mm p ll | C r « 

We expected perceived learning to decline over time as students gained experience with 
computers and the generally positive influence of novelty effects declined (Becker, 1985). Based 
on the gender differences research cited earlier, we also anticipated that girls' perceptions of 
learning would be significantly lower than boys' (Abler & Sedlacek, 1987; Collis & Ollila, 1986; 
Griffin, Gillis, & Brown, 1986; Johnston, 1987; Koohang, 1989; Moore, 1985). THe role of grade 
level was also expected to be significant in that elementary students' perceptions of learning from 
computers would be higher than both middle and high school students' (Krendl, 1986; Smith, 
1987). 

Method 

A panel of 339 respondents randomly selected from all fourth through tenth grade classes in the 
public school system in a small city (population -35,000) in Tennessee participated in the study. ITie 
participants (53.4% female and 46.6% male) completed a self-administered questionnaire at the same 
time each year for three sequential years reporting on their media use habits and their assessments of 
various media activities. 
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Half (50.8%) of the sample reported having used some type of computer (Including video game 
systems) outside of school at the thinning of the study. By the end of the three-year period, 65.2% 
of the students were using computers outside of school. School use, though already well established 
at the outset, further increased during the course of the study from 90% of the students having used 
the technology in at least one class the first year to 97% by the end of the third year. 

Respondents reported that teachers were the most frequent source of help with using a computer 
at all three points in time, probably because most computer use took place in a school setting. Parents 
ranked second in helping, with friends coming in third and siblings last in all three surveys. 
Some specific types of computer use changed during the three-year period of time, while others 
remained stable. For example, video game playing decreased from a high of 72.5% (of the students 
who had used computers) who reported having played such games in the first year to a low of 30.8% 
(of those using computers) by the third year. In contrast, using the computer for word processing 
increased during this period of time from 32.6% in the first wave to 33.7% by the final wave. More 
stability was reflected by such activities «s playing word games (42.9% in 1984 and 38.1% in 1986). 

The following analyses look specifically at students' responses to computers on the three 
dimensions (preference, difficulty, and learning) defining individuals' media attributions. We collected 
data on changes in students' perceptions of the medium as their familiarity with computers increased 
and their uses of the technology changed during the course of the study. Hie nature of these media 
attributions and theii evolution over the three years of the study are examined through a series of 
analyses examining the role of lime, gender and grade level in determining the respondents' attributions 
about the medium over the three years. 

The dependent measures examined in the following analyses are based on students' responses 
to a series of paired comparison judgments about traditional, literacy activities (reading and writing) 
and electronic media activities (using a computer and watching television). Tnese judgments were 



10 

bated on students' comparisons of the activities in terms of the three dimensions idenJfied by dark 
as fundamental to understanding one's attributions and beliefs about meiia activities - which one they 
would prefer in relation to the others, which one they would learn more IWn, and which they would 
find more difficult. Earn activity was compared with all others, resulting in a series of questions at 
follows: 

Which would you rather do ... 

read a book or write 
write or watch teievlaion 
watch television or use a computer 
use a computer or read a book 
read a book or watch television 
write or ute a computer 

Which would be more difficult for you ... 

same at above 
Which would you learn more from ... 

tame as above 

Tnit approach of comparing activities was adopted because activiUes do not exist in isolation; each one 
is only a part of a complex mix of opportunities available to the individual. The ability of each activity 
to fulfill one's needs and desires is assessed relative to other readily available alternatives. 

Thiu indices - one each for preference, difficulty, and learning - indicating bow many times 
each respondent chose the computer over other activities were constructed. These indices served as 
the dependent variables in the following analyses. 

Results 

We used MANOVA repeated measures procedures for longitudinal data (Schaie & Hertzog, 
1981) to assess the role of gender and grade level over time in determining respondents' assessments 
of the computer. Gender and grade level were specified as the between-subject factors and time as the 
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within-subjoctt factor foe subjects' repotted levels of preference, perceived difficulty, tnd perceived 
learning. These analyses address the specific hypotheses examining the role of gender and grade level 
as they relate to computer attributions. 
Computer preference hypotheses 
i Student assessments of computer preference over time confirmed the first hypothesis. Scores 

generally declined over time for the sample as a whole; time 1, M"2.01; time 2, M" 172; time 3, 
M* 1.43. The main effect for time was significant for computer enjoyment: £ (2,332) - 46.87, p. 
< .001. As students became more familiar with the technology, assessments of their enjoyment 
steadily declined. 

Gender and grade level also yielded significant results in relation to computer preference. As 
predicted, girls' scores reflected the same overall pattern as boys'; that is, a steady and monotonic 
decrease over time was apparent. However, despite the similarity in the pattern of the results, girls' 
assessments of computer enjoyment were significantly lower at each point in time than were boys'. 
For girls means were as follows: time 1, M - 1.69; time 2, M ■ 143; time 3, M - 1.06. 
Comparable means for boys were: time 1, M - 2.U; time 2, M - 1.91; time 3, M ■ 1.69. Tne 
main effect for gender was significant: E (1.333) - 51.27, p. < .001. Tnus, the second part of the 
first hypothesis regarding gender differences in assessments of computer enjoyment was confirmed. 

For grade level, the pattern was once again consistent and as predicted. Though their scores 
declined over time, younger students tended to rate the computer as mo»e enjoyable at each point in 
time than did older students. Means by grade level (presented with elementary students first) are as 
follows: time 1, hi * 2.31, 1.91, 1.49; time 2, M - 1.95, 1.65, 1.58; time 3, M » 1.76, 1.30, 
1.07. MANOVA results yielded a significant main effect for grade level: £ (2,333) - 14.79, p. < 
.001. No interaction effects among the independent variables resulted in the analysis. 
Computer difficulty hypotheses 
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We predicted that perceived difficulty with computers would decline over time. This hypothesis 
wis not supported by the analysis. Mesa assessments of difficulty for the whole sample remained 
relatively stable over time: time I, M ■ 2.29; time 2, M ■ 2.39; time 3, M ■ 2.38. The test for 
the main effect of time was not significant: £ (2,332) • 2.11, & • .12. Sttf ttts' perceptions of the 
difficulty of using computers did not decrease with familiarity and experience as anticipated. 

Results for the test of gender effects reflect the pattern established earlier with computer 
preference. That is, the shape of the relationship is similar for boys and girls, with girls being 
significantly different at each point in time. In this case, girls found computers more difficult than 
boys each year, with very little change over the three years of the study. For girls, means were as 
follows: time 1, M B 2.45; time 2, M ■ 2.58; time 3, M ■ 2.50; for boys: time 1J> 2.20; time 
2, M ■ 2.29; time 3, M ■ 2.20. The test for the main effect of gender yielded significant results: 
£ (1,333) * 25.76, g < .001. Once again, then, girls' responses differed significantly from boys', 
with girls perceiving computers to be ccnsisteuly more difficult to use. 

The test of the relationship between grade level and perceived difficulty ' computers supported 
our hypothesis. Elementary students rated the computer as significantly less difficult each year than 
did middle school and high school students. Means (with elementary students listed fust) were as 
follows: time I, M ■ 2.02, 2.44, 2.51; time 2, M - 2.35, 2.37, 2.60; time 3, M - 2.29, 2.44, 
2.33. The main effect of grade level yielded significant results, £ (1,333) ■ 7.56, p < .001. Thus, 
despite the stability of the difficulty ratings over time, the positive relationship between grade level and 
perceived difficulty of using computers proved to be significant. A significant interaction between 
grade level and time also emerged from the analysis, £ (4,666) ■ 4.05, g < .01. What appears to 
happen over time is that the differences in perceived difficulty of using computers tend to converge. 
That is, differences in ratings between younger snd older students are more dispersed at time 1 than 
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at time 3, though elementary students continue to rite the computer as leu difficult at each point in 
time. 

Computer leirntnf hypntht* 

Next we examined tests of the hypotheses related to student assessments of learning from 
computers. The general hypothesis was confirmed. Ratings of learning from computers for the sample 
as a whole declined over time: time 1, M ■ 2.16; time 2, M ■ 1.97; time 3, M ■ 1.82. Th% 
for the main effect of time yielded significant rewlu: £(2,332) ■ 13.79, p < .001. Each additional 
year of experience with the technology resulted in lower student assessments of their perceived learning 
from computers. 

The pattern of the relationship between gender and perceived learning from computers presents 
s similar pattern of results to those reported for previous hypotheses. Thst la, boys and girls tended 
to respond to computers similarly over time, assessments declined over time for both groups, but girls' 
assessments of perceived learning were significantly lower than boys' at each point In time. Means 
for girls were as follows: time 1, M ■ 2.09; time 2, M ■ 1.87; time 3, M ■ 1.75. Tne comparable 
ratings for boys' mean assessments of perceived learning were: time 1, M ■ 2.17; time 2,M ■ 1.98; 
time 3, M ■ 1.81. The main effect for gender was significant, £ (1,333) ■ 3.97, g < .05. These 
results confirmed our expectations regarding the role of gender In relation to perceived learning. 

Analysis of the result* for grsde level also confirmed our expectations. Assessments of 
perceived learning from computers demonstrated a negate relationship with grade level over time. 
Means (beginning with elementary scores) were as follows: time 1, M ■ 2.36, 2.09, 1.97; time 2, 
M « 2.15, 1.89, 1.73; time 3, M ■ 1.97, 1.76, 1.61. The main effect for grade level was 
significant: £ (2,333) » 7.97, g < .001. As demonstrated in the tests for computer preference and 
difficulty, younger students were more optimistic and positive about the computer. Here this Is 
demonstrated by their more generous assessments of the computer's effectiveness as an instructional 
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tool tbaa were older students. They reported higher levels of perceived learning at each pout in time 
than either middle school or high school students. Interactions among thw Independent variables did 
not emerge. 

Discussion 

The purpose of this study was to examine the nature of individuals' responses to computer 
technology over an extended period of time, a time during which computers were rapidly disseminating 
in schools and homes. The orientation adopted focuses on the concept of media attributions as 
identified by Clark (1983; 1984; 1985; Clark & Salomon, 1985). The study responds to criticisms 
levelled against research on computers and learning suggesting that the findings are severely limited, 
invalid, and not generalizable as s result of methodological constraints. Reliance on short-term 
experimental applications of computers using a media comparison approach does not rule out 
alternative explanations, specifically the possible role of novelty effects, in accounting for the generally 
positive results emanating from computer applications. 

This study attempted to overcome these limitations by using a longitudinal panel design, an 
approach that permits the researcher to examine and identify the role of novelty effects as the 
technology becomes established and familiar to users. The results of the study yield support for some 
previous findings and important qualifications for others. The specific areas of interest here focus on 
the evolution of individuals' perceptions of and attributions about computers on the dimensions of 
preference, perceived difficulty and learning. As expected, responses assessing preference for or 
enjoyment of the technology generally tend to decline over time. In addition, both gender and grade 
level play significant roles in determining where these assessments are initially and how much or how 
little they decline during the course of three years. Girls enjoy the technology significantly less than 
boys at each point in time; younger students enjoy computers much more than older students at each 
point in time. 
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Hie result* for perceived difficulty did not confirm our expectations. Individuals* perceptions 
of computer difficulty did not decline as anticipated, tamer they remained quite stable over the three- 
year period. However, once if tin, gender and grade level proved to be important determinants of the 
relttive level of perceived difficulty. Girls reported that computers were significantly more difficult 
than boys at all three measurement points. Older students found computers more difficult to use than 

e 

did younger students all three yean. Tnese results are somewhat surprising. We expected students 
to become more comfortable with the technology as they gained experience with it. This did not 
happen. Computers are viewed as being just as difficult to use at the end of the study as they hat been 
when there were first beginning to be used in schools despite three years of rapid dissemination when 
nearly everyone in the school was using them in classes and home use had increased dramatically. 
Uses of the technology changed during the three years; that is, more word processing was being done, 
for example, and fewer games were being plsyed by the end of the study. However, regardless of 
patterns and types of use, the level of perceived difficulty remained absolutely unchanged. 

Finally, we examined results for perceived learning from computers during the three-year period 
of time. As expected, perceptions of the technology's instructional effectiveness in general declined 
significantly over time. Gender and grade level were sgain significant predictors of perceptions. Girls 
assessed computers as less effective instructional tools than boys. Older students were more skeptical 
about the computer's teaching ability than younger students. This is an intriguing finding given the 
early criticisms of the drill-and-practice orientation of most computer software for in-schoo! 
applications. During the course of the study new and innovative software became available for both 
in-school and out-of-school use, and teachers began to expand their uses of the technology in the 
classroom. Despite the introduction of more sophisticated, challenging, and visually stimulating 
software and teachers' expanded efforts to integrate the technology in to the curriculum, students' 
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perceptions of levnlnf from the technology declined over time, though leu for younger itudenti and 
leu for boyi. 

In terms of future research, these results suggest that there Is strong evidence of novelty effects, 
particularly in the aru of affective responses to the technology end asseuments of its instructional 
effectiveness. Liking and enjoying computers tends to decline with familiarity, u do perceptions of 
its teaching ability. Thus, studies concluding that students like computers, prefer the computer as an 
instructor, and think they learn more from computers than from traditional instructional methods 
following short-term applications are probably misleading. The findings presented herein provide 
evidence that the critics are right - there are problems with the generalizabiiity and validity of earlier 
studies as a result of methodological constraints. It appears that novelty effects do exist. Therefore, 
their presence and influence on findings can no longer be ignored in research on computers and 
learning. 

It is also clear from these results that gender and age are critical factors In understanding 
students' responses to the technology. Gender was identified by Chen (1985) u a consistent factor 
affecting computer attitudes and use m the early research on the technology. This study suggests that 
his conclusion remains valid. The results presented here also suggest that age Is an additional factor 
that consistently shapes individuals' perceptions of and attributions about computers. 

The role of gender and age should be examined in future research on computers. As the 
technology gains a stronger foothold on our educational Institutions and becomes a standard 
instructional tool in the classroom, as well u a fundamental component of cultural literacy, it is critical 
that we understand students' responses to this medium. We know very little about this topic as yet, 
and we know less about how to deal with the associated fears and apprehensions that are experienced 
more by some students than by others. Meanwhile, the number of computers in our schools continues 
to expand, and teachers are becoming more creative in their efforts to integrate the technology into the 
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curriculum. Hie fact that older student! ire leu enamored aad more skeptical about computers and 
that girls are consistently more negative about the technology has profound Implications for the 
instructional potential of the medium. A research agenda for the future should include more and better 
research addressing Issues related to the long-term impact of the technology. Rigorous research on 
this issue would be useful to policy makers, educators, and parents. In addition, we need studies of 
the potential of systematic and thorough training programs to reduce negative affect toward computers 
among certain groups of students. What will probably provide no further benefit to the field are the 
standard small-scale experiments reporting on the short-term effects of computer Implementation. 
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